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1. Introduction - Problems Under Consideration: 

The Department of Political Science at Knox College has long 
been concerned about judging the effectiveness of our teaching and 
counseling of majors. This grant enabled us to survey and/or 
develop a number of tools to the end of identifying strengths and 
weaknesses of our majors and of our program. Data was collected 
during one year on all majors, sophomore through senior years. 

The examination of this data was oriented toward the isolation 
of indicators that would enable us to counsel our students more 
effectively and to understand the complex variables involved in 
the successful completion of the major. The variables utilized * 
in this study are included in pages 8, 9, 10 of this report. 

2. Methods: 

Data was collected on all major's in the department of Poli- 
tical Science during the school year 1969-1970. These variables 
were inter-correlated and subjected to multiple-regression analy- 
sis. A full inter-correlation matrix for these variables is in- 
cluded in the regression process. Since a number of the variables 
were obviously measuring similar or identical phenomena, an infla- 
ted multiple r was produced. The findings are summarized below: 



Population 


Dep. Variable 




R 


r 


All Majors (80) 


Pol. Sci. Cum. 


AVe 


.87 


.76 


Seniors (30) 


Pol. Sci. Cum. 


AVe 


.98' 


.97 




Grad. Rec. Exam 


.89 


.79 


Juniors (22) 


Pol. Sci. Cum. 


AVe 


.94 


.88 


Sophomores (28) 


Pol. Sci. Cum. 


AVe 


.85 


.72 



Standard statistical procedure calls for the squaring of multiple 
r to explain the variance accounted for by the correlation. All 
of the preceding correlations are highly significant, suggesting 
that we have succeeded in defining a number of variables particu- 
larly relfevant to success in the political science major at Knox. 

These tests, however, are cumbersome. In order to develop a 
more efficient set of variables, we decided to eliminate as many 
variables as possible, particularly those that might overlap sig- 
nificantly in measurement. Some variables, such as GRE score for 
instance, were not pertinent to students other than Seniors, and 
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as such could be dropped from the regressions for Sophomores 
and Juniors. Other correlations, such .s sex, were dropped since 
they seemed essentially unrelateable other than by random or chance 
occurence. Our procedure essentially involved systematically 
dropping variables of low correlation with the dependent variable, 
or those variables that would present great difficulty in obtaining 
uncon tarainated data in the future. The resulting regressions are 
presented below. 

Our decision to use the cumulative grade average in the de- 
partment of political science as the dependent variable was based 
on the fact that it is an index available throughout the career 
of the individual major. The most significant regressions com- 
bined with ease of compilation and efficiency of evaluation, uti- 
lized the departmental average as the dependent variable and the 
following variables as independent: SATV, SATM, Options Open 

Scenario 1, Options Open Scenario 2, Grade in Political Science 
201 (introductory course), cumulative departmental evaluations of 
Options/Open of Students, Intensity of Counseling of Students, and 
Graduate Record Examination (seniors only). 



Table of Multiple Regression Coefficients 



Popu la t ion 


Dep. Variable 


R 


2 

r 


Table App. 


All Majors 


l)ep. Average 


.73 


.53 


A 


Seniors 


Dep. Average 


.76 


.58 


B 




GRE 


.74 


.55 


C 


Sophomores 


Dep. Average 


.77 


.60 


D 



3. Findings and Recommendations: 

The multiple correlations are significantly high in all 
cases, suggesting that our elimination of 2/3's of our initial, 
variables still leaves us with variables of sufficient discrimi- 
natory powers. Accordingly, the Department of Political Science 
has developed forms and procedures' whereby these variables can 
be collected, evaluated, and disseminated within the department 
to the end of advising students on an intensive basis. These in- 
struments are enclosed in Appendix F and G. 

On the basis of this study, the Department has concluded 
that certain activities of the Department should be intensified, 
i The intensity of counseling activity is high at this point and 

j should be maintained at that or an even higher level* The evalu- 

! ation of students according to individual professors' assessments 

i of his openess to diverse sources of information and to a variety 

| of alternative political choices should be gathered and made avail 

able to the department as a whole. The importance of the intro- 
ductory course to success in the major is underscored by both its 
intercorrelation with dependent variables and by its effect on 
! the multiple regression when dropped from the calculations (R 
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from .73 to .57). 



Of particular interest is the fact that the multiple r for 
the sophomores was nearly the same as the multiple r for the gra - 
uating seniors, although the intercorrelations vary. This suggests 
that these variables are not only capable of explaining the van 
ance for majors as they complete the prescribed course sequences, 
but that they can serve as well as indicators of the likelihood of 
success in the major as early as the sophomore year. The scenari- 
os seem much more useful on the sophomore level, for instance, 
than on the senior level and should be used in the early identi- 
fication of strength/weakness in majors. The advantages gained 
from having a set of descriptive variables useful, without change 
in scope and definition, for all majors in the department is ap- 
parent. 



We also find it interesting to note that the battery of scales 
included in our initial complex of variables designed to measure 
such psychological dimensions as Political Cynicism, Morale, Faith 
in People, etc., did not contribute greatly to the discriminatory 
powers of the design. They universally exhibited low correlation 
coefficients with those variables we have designated dependent, 
particularly the cumulative average in political science. As 
these are standard tests of some generally recognized sophistication, 
as word to explain their lack of utility is in order. We recog- 
nize two possible limitations on their use in our context. First, 
our population is neither random, nor distributed in any way simi- 
lar to normal population distributions. It is, in short, a popu- 
lation highly skewed in terms of its background characteristics 
and intellectual capabilities. Thus, these tests, intended for 
general use, are not applicable to our particular population. 
Secondly, there is a possible contamination element, deriving from 
the exposure of the population to the theory of such testing, if 
not exposure to those particular tests themselves. Many of the 
tests occur frequently in the journal and professional literature 
of political science, and it is certainly not beyond the realm 
of speculation that a proportion of the subject population would 
be sophisiticated enough to recognize minor modifications of 
these scales. According to these rationale, we have dropped them 
from the compilation of our multiple regressions. 



One effect not included in the computations, but nonetheless 
intriguing and important to our research, was the efC ®ct that our 
testing program had on our majors. It is, of course, di^cult to 
assess with objectivity the complete effect this study had on 
majors, but we subjectively posit a variant of the well-known 
•’Hawthorne Effect.” In this type of situation, the changes made 
are secondary in effect when compared to the impact that knowledge 
that management is interested in staff asserts. In V* J° 81 

that the very fact of our interest in the progress and problems 
of our majors produced a reciprocal concern and increased interest. 
An already strong rapport has intensified and the ease of counseling 
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students, particularly those in difficulty, has increased. We 
suggest, theretore, that the department Lake pains to ensure that 
the majors are aware that our concern is ongoing and directed to 
their eventual successful completion of the major. 

Finally, we wish to comment on the effectiveness of the sce- 
naria devised by the department (see Appendix). Although these 
tests inter-correlated only marginally, they contribute meaning- 
fully to the multiple-regression coefficients. We posit, there- 
fore, that although there is some distinct overlap between the 
tests, that they are for some reason measuring related, but not 
identical, phenomena. We feel that we are on the right track 
in this endeavor and propose in the following year to attempt to 
refine the instruments and apply them to a wider, more randomly 
selected college population. As they attempt to measure psycho- 
logical variables, they are subject to the same possible limitation 
we mentioned in another context above. We particularly wish to 
emphasize that continued research on these scenario involving the 
utilization of "avoidance-avoidance response" - or the tendency 
to avoid making a decision avoidance in character - may prove 
useful. These modified Scenaria will necessitate continued, on- 
going research. They do, however, have the advantage of being 
unpublished and thus the possible level of contamination is re- 
duced. The department, in conjunction with other appropriate 
departments on the campus, will continue the work begun in this 
area. 



In conclusion, the inter-correlation and multiple regression 
of the variables discussed here provided us with the basis for 
eliminating many variables while retaining confidence in the 
following: 



Cumulative Departmental Average, dependent variables 



Scholastic Apt. Test, Verbal 
Scholastic Apt. Test, Math 
Options Open Scenaria, 1 and 2 
Grade in P. S. 201 
Graduate Record Exam 
0/0 Faculty Evaluation 
Average Intensity of 

Counseling (Appraisal) 



independent variable 
independent variable 



»t It 
II II 



II II 



II II 



II II 
II II 



II II 
II II 



II II 



Let us now turn to the instruments the department has developed 
in order to facilitate the continuing use of these variables 
in the evaluation and counseling of majors in the department. 



6 



. l 




O 

ERIC 



8 



Conclusions and Recommendations : 

As reported under "Findings" above, we are satisfied that we have 
identified an important test with high correlation to -ccess^n major- 
• n t -in Political Science at Knox cblle^e. This test* called p 
Open” can and will be refined as w^work with departmental majors over 
the next several years. These testes are included in our Progress Re- 
port. 

Early in our work we became concerned over the amount of record 
keeDine on each student that began to be amassed. It was felt to be 
Zll LTy necessary to keep up to date, clear record, that would be 
usable by each faculty member involved in counseling and to d ® vl 
ti^ schedule when this data could best be recorded and used by the 
faculty We were equally concerned to produce a clearly understood 
graph ^hich could be shown to the student which would exp lam where the 
student's strengths were and where he needed to support weaknesses. A 
ureat deal of difficulty was circumvented when Ur. George Melvi » 

Knox Sector of Institutional Research, devised a new Student Record 
form which so clearly reported a student's standing “ 
ot each term that we could reduce not only the amount but kinds 
records which .we earlier anticipated and prepared 

our proposal. This new form is attached to this report ae Appendix E. 

The new records for our student counseling consequently could be 
simplified for assessing and recording by the Political Science fact . ty. 
This in turn made it possible to chart a studenfs progress more 
and to use the information in counseling. These new forms are attache 

as Appendix F and G. 

One of the advantages of having a chart which can be shown the 
student was first suggested by Knox President Sharvey G. Umbeck. The 
success of tennis notches who could chart all the shots a player makes 
in a game, clearly indicating where and when points 

served clearly as convincing evidence to the player which part of his 

game needed work and strengthening. We acknowledge this charting J 

difficult in a discipline like Political Science, but the effect 
more v . A which can be shown the 

of an accumulation of strengths and weaknesses wnicn can 

student will have a similar effect in convincing him where he needs to 
put “^ effort. Ms intend to alter these charts as experrence rndt- 

cates. 

In order to keep the record keeping current and available to all 
the members of the department, one person will be assigned the P 
aihilitv of setting deadlines and distributing evaluation materials, 

?ie cHUcal p«iSs “r administrative efficiency in record keeping 

attention, much of the difficulty earlier reported can be eliminated. 
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Sophomores , N=28 APPENDIX C - COUNSELING INDICATORS tiple 



APPENDIX D 



Table A: 



Table B 



Table C 
Table D 



APPENDED TABLES A - D 

In tercoixe lational Matrix and Multiple Regression for All 
Majors 



Dependent Variable: Departmental Average R R_ 

Independent Variables: SAT-Verbal 0.177 

SAT-Math 0.168 

0/0-Baliard 0,184 

0/0-Seibert 0.023 

P. S. 201 0.579 

0/0 Faculty Evaluation 0.623 

Av. Int. of Counseling 0.113 

Multiple R 0.725 .53 



.Intercorrelation Matrix and Multiple Regression for Seniors 



Dependent Variable: Graduate Record Exam R R 

Independent .Variables; SAT-Verbal 0.125 

SAT-Math 0.172 

0/0-Ballard 0.100 

0/0-Seibert -0.020 

P. S. 201 0.347 

Dept. Average 0.407 

0/0 Faculty Evaluation -0.029 
Av. Int. of Counseling 0.148 

Multiple R 0.74 .55 

: Dependent Variable: Departmental Average 

Multiple. R 0.76 .58 



: Intercorrelation Matrix and Multiple Regression for Sopho- 

mores 



Dependent Variable: .Departmental Average R R_ 

Independent Variables: SAT-Verbal 0.021 

SAT-Math 0.069 

0/0-Ballard 0.205 

0/0-Seibert 0.357 

P. S. 201 0.393 

0/0 Faculty Evaluation 0.649 

Av. Int. of Counseling 0.577 

Multiple R 0.77 .60 



appendix e 



KNOX COLLEGE EDUCATIONAL DEVELOPMENT RECORD 



Adviser: Mr. R. N. Ballard 

Name: 

Date and Place of Birth: 
December 24 1948 
Evanston Illinois 



CEEB 

Verb Math 
636 563 



Date Entered: 9/66 

Achievement Test Scores 
Math 1 Math II For Lang 
597 Latn 65 



Name' and Address of High School:- 
Alvernia High School 
Chicago Illinois 

High School Units 

Eng Fr Ger Lat Russ Sp Hist (Am) Hist (other) 

4.0 0.0 0.0 2.0 0.0 0.0 1.0 2.0 



High School 
Class Rank 
17/287 



1/10 1/4 

1 1 

Alg Plgra 
3.0 



Trig Sogm Civics Soc St Econ Pol Sci Bio Chem Phys 

1 # 0 o.O _ 0.0 _ 0.0 1.0 _1.0_ _1^°_ 

4.5 Total Credits Needed, plus comprhensive exam 
3.43 upperclass grade index 
300 level courses - complete • 

1 more elementary phys. ed. course(s) needed 
Math proficiency - complete 
Language proficiency - complete 
'Sophomore competency - complete 
Distribution Requirements 
Humanities - Complete 

Math-science - 1.0 more course(s) needed 
Social studies - complete 



Courses taken in major area - Political Science 



PS 


201 


Am Natl Govn 


1.0 


A- 


PS 


343 


Cont Soc Theo 


1.0 


A 


Hist 


202-2 


Amer Hist 


1.0 


B+ 


PS 


216 


Fr Pol Sov Un 


1.0 


A 


PS 


241 


Pol Par Pr Gr 


1.0 


A 


Econ 


211 


Capitalism 


1.0 


A 


PS 


204 


Sc Meth Pol Sc 


1.0 


A 


PS 


311 


Internat Re la 


1.0 


B 


Hist 


324 


Germ 1914-45 


1.0 


B+ 


PS 


202 


St Reg Local 


1.0 


A 


PS 


351 


Indep Studies 


1.0 


A* 


PS 


321 


Am Natl Adm 


1.0 


A- 


PS 


322 


Probs in Adm 


1.0 


* 


PS 


350 


Methods Lab 


1.0 


* 




3.75 


Average in Major 


Area 
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APPENDIX F 
SUMMARY EVALUATION 

N ame Date 

Future Plans: 




4 * Options most open, greatest ability (research, writing, verbal) 

1 *s Options least open, least ability 
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APPENDIX G 

INSTRUCTORS' EVALUATIONS 




H = Houston B = Ballard PH = Haring S = Seibert 
Q s Greenberg 



